How to regulate pattern formations for malware propagation in cyber-physical systems.

Malware propagation can be fatal to cyber-physical systems. How to detect and prevent the spatiotemporal evolution of malware is the major challenge we are facing now. This paper is concerned with the control of Turing patterns arising in a malware propagation model depicted by partial differential equations for the first time. From the control theoretic perspective, the goal is not only to predict the formation and evolution of patterns but also to design the spatiotemporal state feedback scheme to modulate the switch of patterns between different modes. The Turing instability conditions are obtained for the controlled malware propagation model with cross-diffusion. Then, the multi-scale analysis is carried out to explore the amplitude equations near the threshold of Turing bifurcation. The selection and stability of pattern formations are determined based on the established amplitude equations. It is proved that the reaction-diffusion propagation model has three types of patterns: hexagonal pattern, striped pattern, and mixed pattern, and selecting the appropriate control parameters can make the pattern transform among the three patterns. The results of the analysis are numerically verified and provide valuable insights into dynamics and control of patterns embedded in reaction-diffusion systems.

Distributed Discrete-Time Convex Optimization With Closed Convex Set Constraints: Linearly Convergent Algorithm Design.

The convergence rate and applicability to directed graphs with interaction topologies are two important features for practical applications of distributed optimization algorithms. In this article, a new kind of fast distributed discrete-time algorithms is developed for solving convex optimization problems with closed convex set constraints over directed interaction networks. Under the gradient tracking framework, two distributed algorithms are, respectively, designed over balanced and unbalanced graphs, where momentum terms and two time-scales are involved. Furthermore, it is demonstrated that the designed distributed algorithms attain linear speedup convergence rates provided that the momentum coefficients and the step size are appropriately selected. Finally, numerical simulations verify the effectiveness and the global accelerated effect of the designed algorithms.

CM-GAN: A Cross-Modal Generative Adversarial Network for Imputing Completely Missing Data in Digital Industry.

Multimodal data fusion analysis is essential to model the uncertainty of environment awareness in digital industry. However, due to communication failure and cyberattack, the sampled time-series data often have the issue of data missing. In some extreme cases, part of units are unobservable for a long time, which results in complete data missing (CDM). To impute missing data, many models have been proposed. However, they cannot address the CDM issue, because no observation data of the unobservable units can be obtained in this case. Thus, to address the CDM issue, a novel cross-modal generative adversarial network (CM-GAN) is proposed in this article. It combines the cross-modal data fusion technique and the deep adversarial generation technique to construct a cross-modal data generator. This generator can generate long-term time-series data from widely existing spatio-temporal modal data in modern industrial system, and then impute missing value by replacing them with generated data. To test the performance of CM-GAN, extensive experiments are conducted on photovoltaic (PV) power output dataset. Compared with other baseline models, the performance of CM-GAN is generally better and reaches the state-of-the-art level. Moreover, sufficient ablation studies are conducted to present the contribution of the cross-modal data fusion technique and show the reasonability of parameter settings of CM-GAN. Apart from this, some prediction experiments are also conducted. The results show that the PV data recovered by CM-GAN can provide more predictability information for improving the prediction accuracy of deep learning model.

A Collaborative Neurodynamic Optimization Approach to Distributed Nash-Equilibrium Seeking in Multicluster Games With Nonconvex Functions.

In this article, we propose a collaborative neurodynamic optimization (CNO) method for the distributed seeking of generalized Nash equilibriums (GNEs) in multicluster games with nonconvex functions. Based on an augmented Lagrangian function, we develop a projection neural network for the local search of GNEs, and its convergence to a local GNE is proven. We formulate a global optimization problem to which a global optimal solution is a high-quality local GNE, and we adopt a CNO approach consisting of multiple recurrent neural networks for scattering searches and a metaheuristic rule for reinitializing states. We elaborate on an example of a price-bidding problem in an electricity market to demonstrate the viability of the proposed approach.

Learning Attentional Communication with a Common Network for Multiagent Reinforcement Learning.

For multiagent communication and cooperation tasks in partially observable environments, most of the existing works only use the information contained in hidden layers of a network at the current moment, limiting the source of information. In this paper, we propose a novel algorithm named multiagent attentional communication with the common network (MAACCN), which adds a consensus information module to expand the source of communication information. We regard the best-performing overall network in the historical moment for agents as the common network, and we extract consensus knowledge by leveraging such a network. Especially, we combine current observation information with the consensus knowledge to infer more effective information as input for decision-making through the attention mechanism. Experiments conducted on the StarCraft multiagent challenge (SMAC) demonstrate the effectiveness of MAACCN in comparison to a set of baselines and also reveal that MAACCN can improve performance by more than 20% in a super hard scenario especially.

Distributed Multiagent Reinforcement Learning With Action Networks for Dynamic Economic Dispatch.

A new class of distributed multiagent reinforcement learning (MARL) algorithm suitable for problems with coupling constraints is proposed in this article to address the dynamic economic dispatch problem (DEDP) in smart grids. Specifically, the assumption made commonly in most existing results on the DEDP that the cost functions are known and/or convex is removed in this article. A distributed projection optimization algorithm is designed for the generation units to find the feasible power outputs satisfying the coupling constraints. By using a quadratic function to approximate the state-action value function of each generation unit, the approximate optimal solution of the original DEDP can be obtained by solving a convex optimization problem. Then, each action network utilizes a neural network (NN) to learn the relationship between the total power demand and the optimal power output of each generation unit, such that the algorithm obtains the generalization ability to predict the optimal power output distribution on an unseen total power demand. Furthermore, an improved experience replay mechanism is introduced into the action networks to improve the stability of the training process. Finally, the effectiveness and robustness of the proposed MARL algorithm are verified by simulation.

Distributed Actor-Critic Algorithms for Multiagent Reinforcement Learning Over Directed Graphs.

Actor-critic (AC) cooperative multiagent reinforcement learning (MARL) over directed graphs is studied in this article. The goal of the agents in MARL is to maximize the globally averaged return in a distributed way, i.e., each agent can only exchange information with its neighboring agents. AC methods proposed in the literature require the communication graphs to be undirected and the weight matrices to be doubly stochastic (more precisely, the weight matrices are row stochastic and their expectation are column stochastic). Differently from these methods, we propose a distributed AC algorithm for MARL over directed graph with fixed topology that only requires the weight matrix to be row stochastic. Then, we also study the MARL over directed graphs (possibly not connected) with changing topologies, proposing a different distributed AC algorithm based on the push-sum protocol that only requires the weight matrices to be column stochastic. Convergence of the proposed algorithms is proven for linear function approximation of the action value function. Simulations are presented to demonstrate the effectiveness of the proposed algorithms.

Asymptotical Neuro-Adaptive Consensus of Multi-Agent Systems With a High Dimensional Leader and Directed Switching Topology.

We study the asymptotical consensus problem for multi-agent systems (MASs) consisting of a high-dimensional leader and multiple followers with unknown nonlinear dynamics under directed switching topology by using a neural network (NN) adaptive control approach. First, we design an observer for each follower to reconstruct the states of the leader. Second, by using the idea of discontinuous control, we design a discontinuous consensus controller together with an NN adaptive law. Finally, by using the average dwell time (ADT) method and the Barbǎlat's lemma, we show that asymptotical neuroadaptive consensus can be achieved in the considered MAS if the ADT is larger than a positive threshold. Moreover, we study the asymptotical neuroadaptive consensus problem for MASs with intermittent topology. Finally, we perform two simulation examples to validate the obtained theoretical results. In contrast to the existing works, the asymptotical neuroadaptive consensus problem for MASs is firstly solved under directed switching topology.

Discrete-Time Algorithms for Distributed Constrained Convex Optimization With Linear Convergence Rates.

In this article, the constrained optimization problem with its global objective function being the sum of convex local cost functions and the constraint being a closed convex set is researched. The aim of this study is to solve the researched problem in a distributed manner, that is, using only local computations and local information exchanges. Toward this end, two gradient-tracking-based distributed optimization algorithms are designed for the considered problem over weight-balanced and weight-unbalanced graphs, respectively. Since the classical projection method is unsuitable to handle the closed convex set constraint under the gradient-tracking framework, a new indirect projection method is employed in this article to deal with the involved closed convex set constraint. Furthermore, two time scales are introduced to complete the convergence analyses. In addition, under the condition that all local cost functions are strongly convex and L -smooth, it is proved that the algorithms with well-selected fixed step sizes have linear convergence rates.

Observer-Based Consensus Protocol for Directed Switching Networks With a Leader of Nonzero Inputs.

We aim to address the consensus tracking problem for multiple-input-multiple-output (MIMO) linear networked systems under directed switching topologies, where the leader is subject to some nonzero but norm bounded inputs. First, based on the relative outputs, a full-order unknown input observer (UIO) is designed for each agent to track the full states' error among neighboring agents. With the aid of such an observer, a discontinuous feedback protocol is subtly designed. And it is proven that consensus tracking can be achieved in the closed-loop networked system if the average dwell time (ADT) for switching among different interaction graph candidates is larger than a given positive threshold. By using the boundary layer technique, a continuous feedback protocol is skillfully designed and employed. It is shown that the consensus error converges into a bounded set under the designed continuous protocol. Second, as part of the full states' error can be constructed via the agents' outputs, a reduced-order UIO is thus designed based on which discontinuous and continuous feedback protocols are, respectively, proposed. By using the stability theory of the switched systems, it is proven that the consensus error converges asymptotically to 0 under the designed discontinuous protocol, and converges into a bounded set under the designed continuous protocol. Finally, the obtained theoretical results are validated through simulations.

Designing Event-Triggered Observers for Distributed Tracking Consensus of Higher-Order Multiagent Systems.

In this article, the asymptotic tracking consensus problem of higher-order multiagent systems (MASs) with general directed communication graphs is addressed via designing event-triggered control strategies. One common assumption utilized in most existing results on such tracking consensus problem that the inherent dynamics of the leader are the same as those of the followers is removed in this article. In particular, two cases that the dynamics of the leader are subjected, respectively, to bounded input and unknown nonlinearity are considered. To do this, distributed event-triggered observers are first constructed to estimate the state information of the leader. Then, local event-triggered tracking control protocols are designed for each follower to complete the goal of tracking consensus. One distinguishing feature of the present distributed observers lies in the fact that they could avoid the continuous monitoring for the states of the neighbors' observer states. It is also worth pointing out that the present tracking consensus control strategies are fully distributed as no global information related to the directed communication graph is involved in designing the strategies. Two simulation examples are finally presented to verify the efficiency of the theoretical results.

Consensus in High-Power Multiagent Systems With Mixed Unknown Control Directions via Hybrid Nussbaum-Based Control.

This work investigates the consensus tracking problem for high-power nonlinear multiagent systems with partially unknown control directions. The main challenge of considering such dynamics lies in the fact that their linearized dynamics contain uncontrollable modes, making the standard backstepping technique fail; also, the presence of mixed unknown control directions (some being known and some being unknown) requires a piecewise Nussbaum function that exploits the a priori knowledge of the known control directions. The piecewise Nussbaum function technique leaves some open problems, such as Can the technique handle multiagent dynamics beyond the standard backstepping procedure? and Can the technique handle more than one control direction for each agent? In this work, we propose a hybrid Nussbaum technique that can handle uncertain agents with high-power dynamics where the backstepping procedure fails, with nonsmooth behaviors (switching and quantization), and with multiple unknown control directions for each agent.

Output-Feedback Self-Synchronization of Directed Lur'e Networks via Global Connectivity.

In this article, we generalize the results on self-synchronization of Lur'e networks diffusively interconnected through dynamic relative output-feedback from the undirected graph case in Zhang et al. 2016 to the general directed graph case. A linear dynamic self-synchronization protocol of the same structure is adopted as the one proposed in Zhang et al. 2016. That is, the Lur'e-type nonlinearity is not involved in our self-synchronization protocol. It is in fact unknown and only assumed to be incrementally sector bounded within a given sector. In the absence of a leader Lur'e system defining the synchronization trajectory, we construct a novel self-synchronization manifold in order to derive the self-synchronization error dynamics. Meanwhile, the connectivity of the general directed graph having a directed spanning tree is quantified by the global connectivity, instead of the so-called general algebraic connectivity used in the directed graph case under static relative state feedback. The global connectivity plays a crucial role in handling self-synchronization problems of directed nonlinear networks via dynamic relative output feedback, including directed networks with the Lipschitz nonlinear node dynamics, which is also discussed in this article. The protocol parameter matrix design is performed by solving the obtained LMI conditions in sequence. In addition, some discussions are complemented on the important technical details in our self-synchronization protocol design along with extensions. Finally, our theoretical results are illustrated through numerical simulations over a directed nonlinear dynamical network.

Distributed Q-Learning Algorithm for Dynamic Resource Allocation With Unknown Objective Functions and Application to Microgrid.

Dynamic resource allocation problem (DRAP) with unknown cost functions and unknown resource transition functions is studied in this article. The goal of the agents is to minimize the sum of cost functions over given time periods in a distributed way, that is, by only exchanging information with their neighboring agents. First, we propose a distributed Q -learning algorithm for DRAP with unknown cost functions and unknown resource transition functions under discrete local feasibility constraints (DLFCs). It is theoretically proved that the joint policy of agents produced by the distributed Q -learning algorithm can always provide a feasible allocation (FA), that is, satisfying the constraints at each time period. Then, we also study the DRAP with unknown cost functions and unknown resource transition functions under continuous local feasibility constraints (CLFCs), where a novel distributed Q -learning algorithm is proposed based on function approximation and distributed optimization. It should be noted that the update rule of the local policy of each agent can also ensure that the joint policy of agents is an FA at each time period. Such property is of vital importance to execute the ε -greedy policy during the whole training process. Finally, simulations are presented to demonstrate the effectiveness of the proposed algorithms.

A Separation-Based Methodology to Consensus Tracking of Switched High-Order Nonlinear Multiagent Systems.

This work investigates a reduced-complexity adaptive methodology to consensus tracking for a team of uncertain high-order nonlinear systems with switched (possibly asynchronous) dynamics. It is well known that high-order nonlinear systems are intrinsically challenging as feedback linearization and backstepping methods successfully developed for low-order systems fail to work. Even the adding-one-power-integrator methodology, well explored for the single-agent high-order case, presents some complexity issues and is unsuited for distributed control. At the core of the proposed distributed methodology is a newly proposed definition for separable functions: this definition allows the formulation of a separation-based lemma to handle the high-order terms with reduced complexity in the control design. Complexity is reduced in a twofold sense: the control gain of each virtual control law does not have to be incorporated in the next virtual control law iteratively, thus leading to a simpler expression of the control laws; the power of the virtual and actual control laws increases only proportionally (rather than exponentially) with the order of the systems, dramatically reducing high-gain issues.

Fully Distributed Synchronization of Complex Networks With Adaptive Coupling Strengths.

This article considers the fully distributed leaderless synchronization in a complex network by only utilizing local neighboring information to design and tune the coupling strength of each node such that the synchronization problem can be solved without involving any global information of the network. For an undirected network, a fully distributed synchronization algorithm is presented to adjust the coupling strength of each node based on a simple adaptive law. When the topology of a network is directed, two different types of adaptive algorithms are developed to achieve synchronization in a fully distributed manner, where the coupling strength of each node is designed to be either the sum or product of two non-negative scalar functions. The fully distributed leaderless synchronization of a directed network is investigated in a leader-follower framework, where the leader subnetwork is analyzed by using the techniques from constrained Rayleigh quotients and the follower subnetwork is addressed by employing the properties of nonsingular M -matrices. Simulations are given to illustrate the theoretical results.

On Distributed Implementation of Switch-Based Adaptive Dynamic Programming.

Switch-based adaptive dynamic programming (ADP) is an optimal control problem in which a cost must be minimized by switching among a family of dynamical modes. When the system dimension increases, the solution to switch-based ADP is made prohibitive by the exponentially increasing structure of the value function approximator and by the exponentially increasing modes. This technical correspondence proposes a distributed computational method for solving switch-based ADP. The method relies on partitioning the system into agents, each one dealing with a lower dimensional state and a few local modes. Each agent aims to minimize a local version of the global cost while avoiding that its local switching strategy has conflicts with the switching strategies of the neighboring agents. A heuristic algorithm based on the consensus dynamics and Nash equilibrium is proposed to avoid such conflicts. The effectiveness of the proposed method is verified via traffic and building test cases.

A Hybrid Recursive Implementation of Broad Learning With Incremental Features.

The broad learning system (BLS) paradigm has recently emerged as a computationally efficient approach to supervised learning. Its efficiency arises from a learning mechanism based on the method of least-squares. However, the need for storing and inverting large matrices can put the efficiency of such mechanism at risk in big-data scenarios. In this work, we propose a new implementation of BLS in which the need for storing and inverting large matrices is avoided. The distinguishing features of the designed learning mechanism are as follows: 1) the training process can balance between efficient usage of memory and required iterations (hybrid recursive learning) and 2) retraining is avoided when the network is expanded (incremental learning). It is shown that, while the proposed framework is equivalent to the standard BLS in terms of trained network weights,much larger networks than the standard BLS can be smoothly trained by the proposed solution, projecting BLS toward the big-data frontier.

Fault-Tolerant Adaptive Fuzzy Tracking Control for Nonaffine Fractional-Order Full-State-Constrained MISO Systems With Actuator Failures.

The problem of fault-tolerant adaptive fuzzy tracking control against actuator faults is investigated in this article for a type of uncertain nonaffine fractional-order nonlinear full-state-constrained multi-input-single-output (MISO) system. By means of the existence theorem of the implicit function and the intermediate value theorem, the design difficulty arising from nonaffine nonlinear terms is surmounted. Then, the unknown ideal control inputs are approximated by using some suitable fuzzy-logic systems. An adaptive fuzzy fault-tolerant control (FTC) approach is developed by employing the barrier Lyapunov functions and estimating the compounded disturbances. Moreover, under the drive of the reference signals, a sufficient condition ensuring semiglobal uniform ultimate boundedness is obtained for all the signals in the closed-loop system, and it is proved that all the states of nonaffine nonlinear fractional-order systems are guaranteed to remain inside the predetermined compact set. Finally, two numerical examples are provided to exhibit the validity of the designed adaptive fuzzy FTC approach.

Distributed Time-Varying Optimization of Second-Order Multiagent Systems Under Limited Interaction Ranges.

This article investigates the distributed time-varying optimization problem for second-order multiagent systems (MASs) under limited interaction ranges. The goal is to seek the minimum of the sum of local time-varying cost functions (CFs), where each CF is only available to the corresponding agent. Limited communication range refers to the scenario where the agents have limited sensing and communication capabilities, that is, a pair of agents can communicate with each other only if their distance is within a certain range. To handle such a problem, a new continuous connectivity-preserving mechanism is presented to preserve the connectivity of the considered network. Then, two distributed optimization algorithms are presented to solve the optimization problem with time-varying CFs and time-invariant CFs, respectively. Theoretical analysis and two numerical examples are provided to verify the effectiveness of the methods.